Bernstein also makes the point that government debt is not the same as household or business debt, and that is a valid point to make. There are a few differences of note. The first is borrowing -- government has much better ability to borrow than either business or households, the United States in particular. The size and diversity of the economy is one factor, the difficulty of putting a government into default is another, but also the U.S. has control over the value of its currency. It can devalue its way out of debt if need be, like Iceland did. Households and businesses do not have this luxury -- they are usually on very short leashes with their creditors, with grace periods measured in weeks and months, rather than decades.

Ultimately, the only logical choice in the face of a recession is to increase government...

Preferably, this would put the government into a deficit position, not exacerbate it. Debt is, in and of itself, not inherently bad, and especially not for governments. The intelligent and timely use of debt can help to smooth out the volatility in the economy. The issue is that economics clashes with ideology and politics. Ideally, we would see ideology and politics work within the framework of evidence, but this is not really the case. Debt is not bad, but it does matter when you take out debt, how much you take on, and what you spend it on. Those are the choices that should be driven by the ideology and politics, not the issue of debt altogether.
Works Cited:

Bernstein, J. (2012). Rethinking debt. Democracy Journal. Winter 2012. 71-82.